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EXECUTIVE  SUMMARY 


~~-i  Visibility  and  Management  of  Operating  and  Support  Costs  is  a 
program  initiated  by  the  Office  of  the  Secretary  of  Defense  (OSD) 
in  order  to  ensure  that  each  Military  Department  gathers,  tracks, 
and  computes  operating  and  support  costs  by  weapon  system. 

V AMO SC  II  is  an  Air  Force  management  information  system  which  is 
responsive  to  the  OSD  initiative.  It  uses  information  from 
existing  Air  Force  data  systems  to  satisfy  both  Air  Force  and  OSD 
needs  for  certain  weapon  system  operating  and  support  (0&S) 
costs. 

At  present,  the  VAMOSC  II  system  comprises  three  subsystems: 

(1)  The  Weapon  System  Support  Cost  (WSSC)  system  (D160), 
which  deals  with  aircraft, 

(2)  The  Communications  -  Electronics  (C-E)  system  (D160A) , 
which  deals  with  ground  communications  -  electronics 
equipment , 

(3)  The  Component  Support  Cost  Subsystem  (CSCS)  (D160B)  , 
which  deals  with  subsystems  and  components  for  aircraft. 

The  Component  Support  Cost  System  (CSCS)  of  VAMOSC  II 
gathers  and  i  umputes  support  costs  by  assembly /subassembly  and 
relates  those  costs  back  to  the  end  item  or  weapon  system.  CSCS 
replaces  the  Logistic  Support  Cost  (LSC)  model  of  K051  (AFLCR 
400-49)  for  aircraft  and  engines. 

The  CSCS  receives  inputs  from  15  Air  Force  data  systems.  On 
a  quarterly  basis,  the  system  provides  two  standard  reports  each 
processing  cycle  and  twelve  other  types  of  reports  as  requested 
by  users.  It  also  provides  pre-programmed  data  base  extracts  on 
magnetic  tape  on  a  one-time  basis  in  response  to  user  requests. 


Special  requests  for  data  in  user  selected  format  may  also  be 
satisfied  on  a  case  by  case  basis. 

At  the  heart  of  the  CSCS  is  a  set  of  30  algorithms  for  esti¬ 
mation  or  allocation  of  costs.  Information  Spectrum,  Inc.  (ISI) 
was  awarded  a  contract  to  validate  these  algorithms.  This  effort 
included  investigations  of  logic,  appropriateness  of  the 
algorithms  and  assumptions  inherent  in  the  algorithms.  ISI  was 
also  to  survey  published  findings,  reports  of  audit,  etc. 
relating  to  the  accuracy  of  the  source  data  systems.  In  addition 
to  the  algorithm  validation,  ISI  was  to  perform  certain  "special 
tasks,"  including  a  user  survey. 

This  report  provides  the  verification  and  validation  of  the 
algorithm  called  "Base  Direct  Material  Costs."  It  addresses  the 
costs  of  consumable  material  issued  by  base  supply  organizations 
to  maintenance  shops  for  repairs  of  aircraft. 

Supply  organizations  of  the  Air  Force  maintain  records  in 
terms  of  National  Stock  Number,  not  Work  Unit  Code.  In  order  to 
assign  the  costs  of  the  material  to  subsystems  and  components,  an 
allocation  procedure  was  needed.  This  algorithm  allocates  costs 
in  proportion  to  the  number  of  repair  actions  reported. 

In  order  to  verify  and  validate  the  CSCS  algorithms,  a  set  of 
analysis  procedures  applicable  to  all  of  the  algorithms  was 
established.  These  procedures  were  than  applied  to  each 
algorithm.  This  report  first  describes  the  analysis  procedures, 
without  reference  to  the  specific  algorithm  addressed  by  this 
report. 
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Next,  the  Direct  Material  Cost  algorithm  is  defined  and 
described  in  detail.  This  description  includes  identification 
source  data  systems  and  files,  and  the  calculation  procedures 
currently  implemented  by  the  CSCS. 

Finally,  a  critique  of  the  algorithm  is  provided  as  required 
by  the  contract.  It  addresses  the  following  topics: 

o  Verification  of  assumptions  and  approximations 
for  appropriateness  and  accuracy. 

o  Validation  of  accuracy  of  source  data. 

o  Validation  of  appropriateness  of  source  data 
as  inputs  to  CSCS  logic. 

o  Investigation  of  accuracy  and  appropriateness  of  algorithms. 

o  Consideration  of  replacement  of  indirect  cost 
methods  with  more  direct  ones. 

o  Identification  of  algorithm  impact  on  CSCS  output  reports. 
For  each  algorithm  addressed,  ISI  is  required  to  affirm  the  pro¬ 
cess  or  procedure  and  reject  any  portion  that  cannot  be  affirmed. 
Where  the  algorithm  or  portion  of  the  algorithm  is  rejected,  an 
alternate  procedure  must  be  specified. 

No  defects  in  the  Base  Material  Cost  algorithm  could  be  found. 
It  is  recommended  that  it  be  retained  in  its  present  form. 

Also,  as  a  result  of  our  investigations,  personnel  of  the  Air 
Force  Data  System  Design  Center  identified  a  programming  error  in 
a  system  providing  inputs  to  this  algorithm.  The  impact  of  this 
error  is  small.  It  should  be  easy  to  correct,  and  a  recommen¬ 
dation  to  this  effect  is  provided. 
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1 . 0  INTRODUCTION 


Visibility  and  Management  of  Operating  and  Support  Costs  is 
a  program  initiated  by  the  Office  of  the  Secretary  of  Defense 
(OSD)  in  order  to  ensure  that  each  Military  Department  gathers, 
tracks,  and  computes  operating  and  support  costs  by  weapon  system 
(all  costs  are  computed  and  portrayed  in  "then  year"  dollars). 

V AMO SC  II  is  an  Air  Force  management  information  system  which  is 
responsive  to  the  OSD  initiative.  It  uses  information  from 
existing  Air  Force  systems  to  satisfy  both  Air  Force  and  OSD 
needs  for  certain  weapon  system  operating  and  support  (O&S) 
costs. 

At  present,  the  VAMOSC  II  system  comprises  three  subsystems: 

(1)  The  Weapon  System  Support  Cost  (WSSC)  system  (D160), 
which  deals  with  aircraft, 

(2)  The  Communications  -  Electronics  (C-E)  system  (D160A) , 
which  deals  with  ground  communications  -  electronics 
equipment , 

(3)  The  Component  Support  Cost  Subsystem  (CSCS)  (D160B) , 
which  deals  with  subsystems  and  components  for  aircraft. 

1.1  The  Component  Support  Cost  System 

The  Component  Support  Cost  System  (CSCS)  of  VAMOSC  II  gathers 
and  computes  support  costs  by  assembly/subassembly  and  relates 
those  costs  back  to  the  end  item  or  weapon  system.  CSCS  replaces 
the  Logistic  Support  Cost  (LSC)  model  of  K051  (AFLCR  400-49)  for 
aircraft  and  engines. 

The  objectives  of  the  Component  Support  Cost  System  are: 
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(1)  To  improve  the  visibility  of  aircraft  and  engine  com¬ 
ponent  support  costs  and  to  relate  those  costs  to  the 
end  item  or  weapon  system. 

(2)  To  improve  the  Life  Cycle  Costing  capability  for  the 
Air  Force  and  the  Department  of  Defense  in  the 
acquisition  of  new  weapon  systems. 

(3)  To  assist  in  the  design  of  new  weapon  systems  by  pro¬ 
viding  cost  information  on  components  for  existing 
weapon  systems  thereby  enhancing  design  tradeoff  stu¬ 
dies. 

(4)  To  provide  historical  cost  information  at  the  subsystem 
and  component  level  to  improve  logistic  policy  decisions. 

(5)  To  identify  system  component  reliability,  effec¬ 
tiveness,  and  costs  so  that  high  support  cost  items  may 
be  identified  and  addressed. 

The  CSCS  is  described  in  detail  in  references  [1],  [2],  and 
[3].  It  receives  inputs  from  14  Air  Force  data  systems.  On  a 
quarterly  basis,  the  system  provides  two  mandatory  reports  each 
processing  cycle  and  twelve  other  types  of  reports  as  requested 
by  users.  It  also  provides  pre-programmed  data  base  extracts  on 
magnetic  tape  on  a  one-time  basis  in  response  to  user  requests. 
Special  requests  for  data  in  user  selected  format  may  also  be 
satisfied  on  a  case  by  case  basis. 

The  twelve  reports  maintained  above  are  of  primary  interest 
to  the  user  community.  They  are  identified  by  name  in  Table  1. 
Descriptions  and  samples  are  provided  by  reference  [1]. 


2 


•  i 


NUMBER* 

8105 
8104 

8106 

8107 
8111 

8108 

8109 

8110 
8112 

8113 

8114 

8115 


TABLE  1.  CSCS  OUTPUT  REPORTS 


Name 


Cost  Factors 

MDS  Logistics  Support  Costs 

Base  Work  Unit  Code  (WUC)  Costs 

Total  Base  Work  Unit  Code  (WUC)  Costs 

Depot  On-Equipment  Work  Unit  Code  (WUC)  Costs 

Total  Base  and  Depot  Work  Unit  Code  (WUC)  Costs 

NSN— MDS-WUC  Cross-Reference 

MDS-WUC-NSN  Cross-Reference 

Logistic  Support  Cost  Ranking,  Selected  Items 
Summary  of  Cost  Elements 
NSN-WUC  Logistics  Support  Costs 
Assembly-Subassembly  WUC  Costs 


*CSCS  output  reports  are  assigned  Report  control  Symbol 
HAF-LEY  (AR)nnnn,  where  nnnn  is  the  number  in  the  table 
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TABLE  2 .  CSCS 


1.  Base  TCTO  Labor  Cost 

2.  Base  TCTO  Overhead  Cost 

3.  Base  TCTO  Material  Cost 

4.  TCTO  Transportation  Cosi 

5.  Base  Inspection  Costs 

6.  Base  Other  Support  Gene: 

7.  Base  Labor  Costs 

8.  Base  Direct  Material  Cot 

9.  Base  Maintenance  Overhe^ 

10.  Second  Destination  Tram 

11.  Second  Destination  Tram 

12.  Base  Exchangeable  Repai: 

13.  Base  Exchangeable  Repai: 

14.  Base  Exchangeable  Modif; 

15.  Base  Condemnation  Sparei 

16.  Base  Exchangeable  Modif 

17.  Base  Supply  Management  < 

18.  Depot  TCTO  Labor  Costs 

19.  Depot  TCTO  Material  Cosi 

20.  Depot  TCTO  Other  Costs 

21.  Depot  Support  General  Ci 

22.  Depot  Labor  Costs 

23.  Depot  Direct  Material  C< 

24.  Depot  Other  Costs 

25.  Depot  Exchangeable  Repa: 

26.  Depot  Exchangeable  Repa: 

27.  Depot  Exchangeable  Modi! 

28.  Depot  Exchangeable  Modii 

29.  Depot  Condemnation  Span 

30.  Depot  Material  Managemei 


2.0  ANALYSIS  PROCEDURES 


In  order  to  verify  and  validate  the  CSCS  algorithms,  a  set 
of  analysis  procedures  applicable  to  all  of  the  algorithms  was 
established.  These  procedures  were  then  applied  to  each 
algorithm.  This  section  describes  the  analysis  procedures 
without  reference  to  the  specific  algorithm  addressed  by  this 
report. 

The  algorithm  analysis  process  consists  of  six  portions, 
described  in  the  following  sections. 

2.1  Algorithm  Description 

The  algorithms  are  described  in  references  [1],  [2],  and  [3]. 
These  descriptions  are  not  identical.  In  general  they  supple¬ 
ment,  rather  than  contradict  each  other.  The  first  two  describe 
what  the  system  is  to  achieve;  the  third  describes  the  system 
design  to  do  so. 

None  of  these  descriptions  provides  the  combination  of  level 
of  detail  and  clarity  of  concept  required  for  this  validation 
effort.  The  first  step  in  the  analysis  methodology  was  the 
generation  of  such  a  description.  The  descriptions  in  the  three 
reference  sources  just  cited  were  studied.  Assumptions  about 
data  processing  procedures  were  made  explicit.  When  necessary. 
Air  Force  personnel  involved  in  implementation  of  the  D160B  sub¬ 
system  were  contacted  for  clarification. 

2.2  Input  Data  Definitions 

Closely  related  to  the  first  step  was  the  clarification  of 
the  definitions  of  the  input  data.  The  identification  of  each 
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input  data  element  and  of  the  system  providing  it  was  provided  by 
the  User's  Manual  (reference  [1]).  This  identification  was 
refined  by  identification  of  a  particular  file  within  the  source 
system  and  the  structure  of  the  file  as  described  in  both  the  CSCS 
System/Subsystem  Specification  and  in  the  Memoranda  of  Agreement. 
The  Memoranda  of  Agreement  have  been  established  between  the 
Office  of  V AMO SC  and  the  Offices  of  Primary  Responsibility  (OPR) 
for  the  systems  providing  the  input  data.  Any  inconsistencies  or 
voids  were  identified  and  resolved  through  contact  with  the 
Office  of  V AMO SC  and/or  implementing  personnel. 

Whenever  appropriate,  input  data  element  definitions  were 
further  refined  by  tracing  the  elements  back  to  their  sources 
through  the  reference  data  provided.  If  these  were  inadequate, 
the  OPRs  were  contacted  directly  for  clarifications.  In  tracing 
the  data  back  to  their  origins,  possible  sources  of  data  con¬ 
tamination  were  considered.  Information  on  the  likelihood  and 
significance  of  such  contamination  was  collected  from  cognizant 
personnel  and  from  published  references. 

2.3  Concept  Validation 

The  two  steps  above  established  exactly  what  the  algorithm 
does.  The  third,  and  most  criti.  si  step,  considered  the  validity 
of  the  procedure.  It  depended  on  the  ability  of  the  analyst  to 
translate  mathematical  formulas  and  data  processing  techniques 
into  meaningful  concepts. 

Some  explicit  techniques  which  were  generally  used  in  concept 
validation  for  all  algorithms  are  listed  below. 

(a)  Consider  how  the  cost  element  would  be  calculated  if 
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there  were  no  constraints  on  resources.  (For  example, 
suppose  the  CSCS  could  identify  the  pay  grade -and  hours 
worked  of  each  individual  involved  in  a  maintenance 
action. ) 

(b)  Identify  assumptions*  incorporated  into  the  Algorithm. 
Generally  this  procedure  will  identify  the  real 
constraints  which  affect  the  approach  in  (a)  above. 

(c)  Identify  approximations  incorporated  into  the 
algorithm.  For  instance,  one  such  approximation  is  the 
use  of  an  average  labor  rate  for  each  aircraft. 

(d)  Study  each  approximation  for  possible  sources  of  error. 
Some  examples  are  biases  introduced  by  editing  proce¬ 
dures,  obsolete  data,  or  inappropriate  application. 
Whenever  feasible,  estimate  the  likelihood  of  these 
errors  by  reviews  of  the  literature  and  contact  with 
cognizant  personnel. 

(e)  Test  the  algorithms  under  conditions  of  assumed  extreme 
values  for  the  inputs.  For  instance,  in  evaluating  the 
algorithm  for  base  maintenance  overhead  costs,  assume 
that  for  a  single  reporting  period  all  maintenance 
labor  is  overhead  and  none  is  direct.  Also  try  the 


*NOte  that  assumptions,  approximations,  and  allocations  are 
different  concepts,  although  in  some  cases  the  boundaries 
between  them  are  not  sharp.  ISI  has  recognized  few  assump¬ 
tions  in  the  algorithms,  but  many  approximations  and 
allocations. 


reverse  assumption.  I£  an  assumption  of  an  extreme 
input  leads  to  an  illogical  result,  the  algorithm  is 
flawed.  — 

Task  4  of  Section  C-2  of  the  contract  speaks  of 
appropriate  statistical  techniques  to  confirm  or  repu¬ 
diate  each  algorithm.  Statistical  techniques  could 
confirm  or  repudiate  only  statistical  hypotheses  as 
assumptions.  (Use  of  an  average  does  not  constitute  an 
assumption.)  Accordingly,  statistical  techniques  apply 
to  confirmation  or  repudiation  of  an  algorithm  only  to 
the  extent  that  statistical  hypotheses  can  be 
developed . 

(f)  As  each  algorithm  is  considered,  ensure  that  the  costs 
do  not  overlap  others  already  accounted  for.  (In  some 
cases  an  overlap  may  be  necessary  and  desirable.  Where 
this  occurs,  the  overlap  will  be  noted.) 

(g)  In  each  CSCS  output  report,  identify  the  data  elements 
incorporating  the  output  of  the  algorithm,  so  that  a 
final  assessment  of  report  accuracy  can  be  made  for 
each  output  report. 

(h)  Consider  alternative  sources  of  input  data  for  the 
algorithm.  Also  consider  more  direct  cost  assignments 
then  those  incorporated  in  the  algorithm. 

2.4  Problem  Resolution 

Whenever  a  significant  deficiency  was  recognized  in  one  of 
the  algorithms,  one  or  more  proposed  solutions  were  developed. 
This  was  a  creative  analytic  process  for  which  few  guidelines 
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could  be  proposed  in  advance.  Certainly  it  depended  on  famil¬ 
iarity  with  the  various  existing  Air  Force  data  reporting  and 
processing  systems.  Proposed  solutions  were  discussed  with  per¬ 
sonnel  of  the  Office  of  VAMOSC,  and  revised  as  appropriate. 
Recommended  solutions  were  expressed  in  the  form  of  contributions 
to  a  draft  Data  Automation  Requirement  (DAR)  when  these  would  be 
applicable. 

2.5  Documentation 

The  documentation  of  the  analysis  of  each  algorithm  was  a 
crucial  part  of  the  effort.  Emphasis  was  placed  on  making  it 
thorough,  clear,  and  unambiguous.  In  the  documentation,  every 
assertion  was  substantiated.  This  was  done  by  reference  to 
source  documentation,  by  explicitly  expressed  application  of  the 
experience  and  judgment  of  the  contractor,  or  by  citation  of 
information  provided  by  cognizant  Air  Force  personnel.  In  the 
last  case,  the  information  was  supported  by  documentation  iden¬ 
tifying  the  source,  the  date,  and  the  information  provided. 

3.0  ALGORITHM  ANALYSIS 

The  previous  section  described  the  general  analysis  proce¬ 
dures  applied  to  all  algorithms.  This  section  presents  the 
results  of  applying  those  procedures  to  the  algorithm  for  Base 
Direct  Material  Costs. 

Section  3.1  Provides  a  detailed  description  of  the  algorithm 
and  of  the  input  data  it  uses.  Section  3.2  provides  a  critique, 
structured  to  correspond  to  the  contractual  requirements. 

Section  4.0  makes  recommendations  for  solutions  of  problems. 
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3.1  Algorithm  Description 


In  the  following  description  COBOL-type  data  names. are  used 
to  express  the  algorithm  output  and  its  components.  The  available 
source  documentation  does  not  provide  the  actual  data  names  used 
by  the  CSCS  programs.  They  are  presumably  different  from  those 
used  in  this  report. 

This  description  provides  a  formula  for  the  calculation  that 
is  derived  from  the  Users  Manual  and  other  sources.  It  is  not 
the  same  as  the  formula  provided  in  the  Users  Manual.  It  is 
intended  to  be  more  explicit.  The  formula  is  stated  in  Section 
3.1.1.  The  input  data  elements  and  their  sources  are  provided  in 
Section  3.1.2.  The  calculation  is  described  verbally  in  Section 
3.1.3.  Unless  otherwise  noted,  the  descriptions  are  based  on 
references  [1],  [2],  and  [31,  and  on  direct  discussion  with  per¬ 
sonnel  of  the  Office  of  VAMOSC.  In  case  of  any  discrepancies, 
information  provided  by  knowledgeable  personnel  was  accepted  as 
most  current,  hence  most  definitive. 

3.1.1  Calculations 

In  this  report  the  calculations  are  explained  with  inter¬ 
mediate  steps  displaying  the  aggregation  of  counts  of  repair 
actions. 

(a)  MDS-BASE-WUC-REP-ACTS  -  MDS-BASE-WUC-ACTS-ON 

+  MDS-BASE-WUC-REP-ACTS-OFF. 

(b)  MDS -BASE- SYS -REP -ACTS  «  £g  MDS-BASE-WUC-REP-ACTS, 

where denotes  summation  over  all  repair  action  reports 
for  the  designated  system. 

(C)  MDS-BASE-SRD -REP -ACTS  - Z MDS-BAS E- SYS- REP- ACTS , 

where  Z  denotes  summation  over  all  systems  for  the  entire 
aircraft. 
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(d)  MDS-BASE-WUC-MAT-COST 


-  MDS-BASE-SYS-MAT-COST  x  MDS-BASE-WUC-REP-ACTS 

MDS-BASE-SYS-REPjACTS 

+  MDS-BASE-SRD-MAT-COST  x  MDS-BASE-WUC-REP-ACTS 

MDS-BASE-SRD- REP -ACTS 

3.1.2  Inputs 

Name:  MDS-BASE-WUC-REP-ACTS-ON 

Definition:  Number  of  on-equipment  repair  action  reports 
for  the  calendar  quarter,  base,  MDS,  and  five 
digit  Work  Unit  Code. 

Source  System/File:  D056A/MNI7OKO 


Name:  MDS-BASE-WUC-REP-ACTS-OFF 

Definition:  Number  of  off-equipment  repair  actions  reports 
for  the  calendar  quarter,  base  MDS,  and  five 
digit  Work  Unit  Code. 

Source  System/File:  D056C/MPI15KO 


NAME:  MDS-BASE-SYS-MAT-COST 

Definition:  Total  of  currently  listed  prices  for  all 

material  issued  at  the  base  for  the  calendar 
quarter  for  the  MDS  (identified  by  SRD)  and 
system  (identified  by  two  digit  WUC). 

Source  System/File:  DO 02 A/ ID 


Name:  MDS-BASE-SRD-MAT-COST 

Definition:  Total  of  currently  listed  prices  for  all  material 
issued  at  the  base  for  the  calendar  quarter  for 
MDS  (identified  by  SRD)  but  lacking  iden¬ 
tification  for  the  system  to  which  it  applies. 

Source  System/File:  D002A/<D 


(l)»Base  Consumable  material"  file  (no  number). 


3.1.3  Description  of  Calculation  Procedure 


As  indicated  in  Section  3.1.2,  on  equipment  actions  and  off- 
equipment  actions  are  passed  to  the  CSCS  monthly  by  two  different 
parts  of  DO 56.  In  step  (a)  of  Section  3.1.1,  the  on-equipment 
and  off-equipment  repair  action  counts  are  needed  for  each 
individual  Work  Unit  Code  (system)  on  the  aircraft.  The  sum  of  the 
five  digit  repair  action  counts  is  calculated  in  step  (b).  These 
counts  are  added  for  all  systems  on  the  aircraft  in  step  (c), 
yielding  a  total  count  for  the  aircraft. 

Prom  the  Automated  Material  System  interfaced  with  the  Supply 
System  at  Base  Level  (D002A),  the  CSCS  receives  monthly  reports 
of  all  material  issued  for  the  MDS  and  base.  Some  of  these  reports 
identify  the  system  to  which  the  material  applies,  and  others  do 
not.  The  reports  include  the  current  price  for  each  issued  item. 

The  CSCS  accumulates  the  prices  for  the  quarter  for  each  system 
when  one  is  identified,  and  for  the  whole  aircraft  when  one  is  not. 

Step  (d)  of  Section  3.1.1  achieves  the  allocation  of  material 
costs  to  the  five  digit  Work  Unit  Codes.  A  single  Work  Unit  Code 
is  assigned  a  portion  of  the  system  material  costs  for  the  system 
to  which  it  belongs,  as  well  as  a  portion  of  the  "whole  aircraft” 
costs.  One  way  to  look  at  calculation  (d)  is  that  system 
material  costs  are  divided  by  the  number  of  system  repair  actions, 
yielding  average  system  material  costs  per  repair  action.  Also, 
"whole  aircraft”  material  costs  are  divided  by  total  repair 
actions,  yielding  average  "whole  aircraft”  material  costs  per  repair 
action.  Then,  for  each  Work  Unit  Code,  the  number  of  repair 
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actions  for  the  quarter  is  multiplied  by  the  sum  of  both  kinds  of 
costs.  This  explanation  corresponds  to  rewriting  formula  (d)  as 


(d)  MDS-BASE-WUC -MAT -COST 

, /mds-base-sys-mat-cost  mds-base-srd-mat-cost \ 

\MDS-BASE-SYS-REP-ACTS  +  MDS-BASE-SRD-REP-ACTS/ 

X  MDS-BASE-WUC-REP-ACTS 

3-2  Critique  of  Algorithm 

This  section  addresses  various  facets  of  the  algorithm.  The 
discussion  is  structured  to  correspond  to  the  contractual  require¬ 
ments.  Each  aspect  is  either  affirmed  or  rejected.  Rejections 
lead  to  recommendations  in  Section  4.0. 

3.2.1  Appropriateness  and  Accuracy  of  Assumptions  and 
Approximations 

The  algorithm  is  based  on  one  approximation:  that  the  costs 
of  materials  used  in  repairs  is  proportional  to  the  number  of 
maintenance  actions.  This  approximation  is  invoked  to  allocate 
costs  to  the  five  digit  Work  Unit  Codes  from  the  two  digit  or 
whole  aircraft  level,  depending  on  which  way  they  are  reported. 

Informal  discussions  with  Air  Force  personnel  show  that  it  is 
generally  believed  that  in  most  cases  a  maintenance  event  yields 
a  single  maintenance  action.  A  "maintenance  event"  technically 
corresponds  to  the  assignment  of  a  job  control  number. 

Intuitively,  a  maintenance  event  corresponds  to  a  response  to  an 
apparent  malfunction  at  either  the  organizational  or  intermediate 
level . 


Since  materials  issues  are  not  identified  with  five  digit 
Work  Unit  Codes  by  current  reporting  systems,  some  allocating 
scheme  is  certainly  needed.  ISI  can  identify  three  possible 
methodologies  for  allocation  based  on  the  following  proxies: 
number  of  maintenance  actions,  number  of  maintenance  events,  or 
maintenance  man-hours.  We  can  find  no  basis  for  preferring 
either  of  the  two  to  the  number  of  maintenance  actions. 
Accordingly,  we  affirm  the  appropriateness  of  this  choice. 

As  for  accuracy,  it  is  easy  to  see  that  the  sum  of  the  values 
given  by  formula  (d)  of  Section  3.1.2  for  all  Work  Unit  Codes  on 
one  aircraft  MDS  is  the  sum  of  all  costs  provided  by  D002A  for 
that  MDS.  Hence  the  allocations  are  exclusive  and  exhaustive  of 
the  material  costs.  Together  with  appropriateness,  this  is  all 
we  can  ask  of  an  allocation  procedure.  Exact  values  are  not  a 
criterion,  since  if  they  were  available,  no  approximation  would 
be  needed.  ISI  affirms  the  accuracy  of  the  approximation  as  well 
as  its  appropriateness. 

3.2.2  Accuracy  of  source  Data  and  Congruence  of  Data  Element 
Definitions 

Information  Spectrum  was  directed  to  validate  accuracy  of 
source  data  based  on  a  survey  of  published  findings,  reports  of 
audit,  etc.  No  direct  sampling  of  data  was  to  be  performed.  The 
Office  of  V AMO SC  has  indicated  that  direct  validation  of  source 
data  is  planned  for  future  efforts 

This  algorithm  receives  data  from  two  systems:  the  Product 
Performance  System  (D056)  and  the  Automated  Material  System 


Interfaced  with  Supply  Systems  at  Base  Level  (D002A).  No 
published  criticism  of  the  accuracy  of  the  latter  could -be  found, 
and  ISI  affirms  its  accuracy.  The  accuracy  of  the  D056-system 
has  been  criticized,  notably  in  reference  [11].  The  criticisms 
are  reviewed  in  reference  [17].  That  reference  also  points  out 
that  a  new  maintenance  data  collection  system  under  development, 
the  Automated  Maintenance  System  (AMS),  holds  considerable  pro¬ 
mise  of  improvement. 

In  any  event,  the  impact  of  any  inaccuracies  in  the  0056 
system  in  this  algorithm  is  lessened  because  of  the  way  the  data 
from  that  system  is  used.  In  this  algorithm,  the  D056  data  is 
the  basis  for  the  allocation  procedure.  Absolute  errors  in  the 
D056  reports  have  no  effect  on  the  algorithm.  There  are  adverse 
effects  only  to  the  extent  that  maintenance  actions  are  under  (or 
over)  reported  for  one  Work  Unit  Code,  or  for  one  system,  as  com¬ 
pared  with  others,  at  a  single  base  and  for  a  single  MDS.  It  is 
reasonable  to  expect  that  any  such  biases  should  be  much  smaller 
than  the  absolute  errors  in  man-hour  reporting.  Information 
Spectrum  believes  that  the  impact  of  DO 56  data  errors  on  the  Base 
Direct  Material  Cost  algorithm  is  not  significant. 

The  "congruence  of  data  element  definitions”  considers  whether 
the  source  data  elements  really  mean  what  the  CSCS  algorithm 
implicitly  "thinks"  they  mean.  This  question  requires  some 
detailed  discussions. 
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3. 2. 2.1  Material  Costs 


Material  costs  are  extracted  by  D002A  from  a  file  called  the 
Transaction  History  File.  This  file  has  a  record  for  every 
supply  transaction.  "Transactions"  encompass  a  wide  range  of 
activities,  including  issues,  turn-ins,  receipts,  shipments,  and  a 
variety  of  adjustments.  The  CSCS  is  interested  only  in  issues. 

On  some  occasions  material  is  turned  into  supply  from  maintenance; 
the  CSCS  is  concerned  with  net  issues. 

Issues  and  turn-ins  include  many  transactions  of  no  concern 
to  the  CSCS.  In  order  to  reduce  unnecessary  storage  and  pro¬ 
cessing  of  irrelevant  information,  D002A  uses  record  selection 
criteria  to  choose  records  to  be  passed  to  the  CSCS.  The 
criteria  are  reproduced  in  Attachment  1,  copied  from  reference 
[24].  These  criteria  evolved  from  methods  used  for  other  pur¬ 
poses.  The  rationale  behind  them  is  not  documented.  Information 
Spectrum  is  satisfied,  on  the  basis  of  its  study  of  the  criteria, 
that  the  selection  criteria  do  not  cause  exclusion  of  any  records 
the  CSCS  would  want. 

However,  as  a  result  of  our  investigations,  an  error  in  pro¬ 
cedure  by  D002A  was  discovered.  The  error  was  acknowledged  in 
reference  [24],  which  identified  it  as  follows: 

"Local  manufacture  transactions  (TRIC  "REC"  with 
FIA  440)  are  assumed  to  be  issues  by  the  program 
instead  of  a  turn-in.  The  end  result  is  a  local 
manufacture  receipt  and  an  issue  can  cause  iden- 
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tical  contributions  to  the  Daily  CSCS  Record 
( 1CM ) .  " 

Instead  of  subtracting  local  manufacture  receipts  from  issues, 
D002A  is  adding  them.  This  error  has  little  impact,  since  local 
manufacture  (by  intermediate  maintenance  shops)  is  uncommon,  and 
tends  to  apply  to  low-priced  items. 

Also  of  interest  are  the  unit  prices  used  by  D002A. 

Reference  [24]  states  that  prices  are  updated  through  the  Stock 
Number  User  Directory  (D071),  Contract  (Local  Purchase)  Price 
Changes,  or  manually  prepared  entries.  D071  gets  the  price 
updates  from  D043,  the  Master  Item  Identification  Control  System 
(reference  [25],  Chapter  9,  Section  21).  D043  gets  its  updates 

from  the  Item  Managers  at  the  ALCs  (reference  [26],  Chapter  35). 
Discussions  with  Air  Force  personnel  indicate  that  Item  Managers 
update  prices  when  they  purchase  new  items  for  the  inventory,  but 
may  also  update  as  they  think  appropriate. 

It  may  be  argued  that  the  ideal  unit  price  for  the  CSCS  is 
the  price  at  which  the  issued  item  was  purchased,  or  the  current 
cost  of  replacing  it.  (The  latter  is  not  well  defined,  because  it 
depends  on  the  quantity  purchased. )  The  price  being  used  may  be 
described  as  the  current  "catalog  price."  Information  Spectrum 
considers  this  definition,  as  good  as,  or  better  than,  the  alter¬ 
natives.  It  is  desirable,  however  that  system  users  be  informed 
of  the  meaning  of  the  price. 

3. 2. 2. 2  Number  of  Repair  Actions 

Repair  actions  are  initially  documented  on  AFTO  Form  349, 
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3.2.4  Accuracy  and  Appropriateness  of  the  Algorithm 

It  has  already  been  noted  that  material  costs  are  not 
directly  linked  to  the  five  digit  Work  Unit  Codes  by  existing 
reporting  systems.  Accordingly,  some  approximation  is 
necessary.  Section  3.2.1  showed  that  the  allocation  scheme  does 
account  for  all  material  costs,  and  associates  these  with  five 
digit  Work  Unit  Codes  in  a  reasonable  way.  ISI  affirms  the 
accuracy  and  appropriateness  of  the  algorithm. 

3.2.5  Directness  of  Costing 

By  incorporating  material  transactions  as  reported  at  their 
source,  the  algorithm  accounts  for  transactions  as  directly  as 
possible.  The  conversion  of  transactions  to  costs  in  terms  of 
current  catalog  prices  is  discussed  in  Section  3. 2. 2.1;  this 
approach  may  be  described  as  direct  as  well.  Accordingly,  we 
affirm  the  directness  of  costing  in  this  algorithm. 

3.2.6  Application  to  CSCS  Output  Reports 

Direct  material  costs  are  components  of  CSCS  reports  as 
described  by  Table  3.  The  accuracy  of  the  algorithm  output  will 
impact  the  accuracy  of  the  reports  as  a  whole.  However,  the 
total  report  accuracy  cannot  be  addressed  until  all  algorithms 
are  reviewed.  This  will  occur  in  the  final  report  of  this 
effort.  Evaluation  of  the  usefulness  of  the  report  will  also  be 
provided  in  the  final  report  of  this  effort  and  after  ISI  con¬ 
ducts  a  survey  of  users. 
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OUTPUT  1 


I 


1.  MDS 
Cos' 


2 .  Bas< 
(WU( 


3 .  Toti 
Codi 


4.  Toti 
Worl 
Cosi 


5.  Sunn 
Elei 


(1)  esc: 

(AR 

rep< 

(2)  i dei 


TABLE  3 


CONTRIBUTION  OF  BASE  DIRECT  MATERIAL 
COSTS  ALGORITHM  TO  CSCS  OUTPUT  REPORTS 

COST  ELEMENTS  CONTRIBUTED 

OUTPUT  REPORT  (D  TO  BY  THE  ALGORITHM  * £ l 

1.  MDS  Logistic  Support  1.  By  MDS  for  all  Bases: 

Costs/8104  a.  WUC  Component  Costs 

(1)  Base 

b.  Total  MDS  Costs 

c.  (Two  digit)  WUC  Cost 


2.  Base  Work  Unit  Code  2.  By  MDS  and  Bases: 

(WUC)  Cost/8106  a.  Total  Base  Costs, 

Component 

b.  (Five  digit)  WUC 

Direct  Material  Costs 

c.  (Jive  digit)  Total  WUC 
Costs 


3.  Total  Base  Work  Unit 
Code  (WUC)  Costs/8107 


4.  Total  Base  and  Depot 
Work  Unit  Code  (WUC) 
Costs/8108 


5.  Summary  of  Cost 
Elements/8113 


3.  By  MDS  for  all  Bases: 

a.  Total  Base  Costs, 
Component 

b.  (Five  digit)  WUC  Direct 
Material  Costs 

c.  (Five  digit)  Total  WUC 
Costs 

4.  By  MDS  for  all  Bases: 

a.  Total  Costs,  Component 

b.  (Five  digit)  Base 
Direct  Material  Costs 

c.  Base  and  Depot  (Five 
digit)  WUC  Total 

5.  By  MDS  for  all  Bases: 

a.  Unit  Level  Consumption 
Direct  Material  Costs 


(1)  CSCS  output  reports  are  assigned  Report  Control  Symbol  HAF-LEY 
(AR)  nnnn,  where  nnnn  is  the  number  indicated  in  the  output 
report  title  in  Table  3. 

Identified  by  the  title  printed  in  the  report. 


4.0  RECOMMENDATIONS 


Section  3  has  presented  ISI's  judgement  that  the  algorithm 
for  Base  Direct  Material  Cost  is  valid.  Every  aspect  of  the 
algorithm  was  affirmed,  and  we  recommend  that  it  be  retained  in 
its  present  form. 

Section  3. 2. 2.1  described  a  programming  error  in  the  D002A 
system* 1)  discovered  by  personnel  of  the  Air  Force  Data  Systems 
Design  Center  as  a  result  of  investigations  by  Information 
Spectrum.  Section  4.1  presents  a  recommendation  to  correct  the 
error. 

4.0a  Office  of  VAMOSC  (OOV)  Comments 
Concur 

4.1  Correction  to  D002A 

It  is  recommended  the  programming  error  in  D002A  be  corrected. 
Appropriate  DAR  entries  are  provided  in  attachment  2. 

4. la  Office  of  VAMOSC  (OOV)  Comments 

Concur.  The  DAR  will  be  submitted  in  conjunction  with  a 
request  for  additional  organization  codes  needed  by  the  C-E 
system.  OOV  will  submit  the  DAR  by  29  Feb  1984. 


In  fact  there  were  two  errors  discovered.  The  other  one  is 
addressed  in  reference  [34]. 
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MEMORANDA  OF  AGREEMENT 
FOR  SYSTEM  INTERFACES 


Ref.  No. 

Memorandum  No. 

Date 

[6.1] 

D002A/M024B/D160B-A 

9 

Jun 

1980 

[6.2] 

D002A/M024B/D160B-B 

9 

Jun 

1980 

[6.3] 

D024A/D160B-A 

30 

Jun 

1980 

16.4] 

D033./ARC/D160B 

14 

Jun 

1980 

[6.5] 

D042A/DNB/D160B 

4 

Nov 

1983 

[6.6] 

DO 4 6/MO 2 4 /Dl 6 OB 

9 

Apr 

1981 

[6.7] 

D046/D160B 

23 

Jun 

1982 

[6.8] 

D056A/BDN/D160B-A 

23 

Jan 

1981 

[6.9] 

D056A/D160B-C 

13 

Oct 

1981 

[6.10] 

D056A/D160B-D 

29 

Jan 

1981 

[6.11] 

D056A  F00 5 

25 

Apr 

1979 

[6.12] 

D056B/BDN/D160B-A 

22 

Dec 

1980 

[6.13] 

D056C/D160B-A 

4 

Mar 

1981 

[6.14] 

D071/D160B 

17 

Jun 

1982 

[6.15] 

D143B/D002A  9159 

3 

Aug 

1979 

[6.16] 

D143F/ARC/D160B-A 

5 

Feb 

1981 

[6.17] 

D160/D160B 

11 

Jun 

1982 

[6. 18] 

G004L/M024B/D160B-A 

30 

May 

1980 

[6.19] 

G004L/M024B/D160B-B 

30 

May 

1980 

[6.20] 

G004L/M024B/D160B-C 

5 

Nov 

1981 

[6.21] 

G019F/D160B 

8 

Sep 

1982 

[6.22] 

G033B/D160B 

12 

Jul 

1982 

[6.23] 

G072D/BDN/D160B-A 

19 

Apr 

1982 
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MEMORANDA  OF  AGREEMENT 

FOR  SYSTEM  INTERFACES  (Continued) 

Ref.  No . 

Memorandum  No. 

Date 

[6.24] 

HO  36B/RC/D160B-A 

10 

Feb 

1981 

[6.25] 

H069R/M024B/D160B-B 

19 

Jan 

1981 

[6.26] 

OOl 3/BDN/D160B 

22 

Jul 

1982 
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ATTACHMENT  1 

TRANSACTION  HISTORY 
RECORD  SELECTION  CRITERIA 

(Copied  from  Reference  [24]) 


Al-1 


TRANSACTION  HISTORY  RECORD  SELECTION  CRITERIA 
(Program  990/D17) 


1.  Step  Is  Select  transaction  history  records  with  "Type  OCCR 
Code"  equal  to  7,  8,  and.  Exclude  the  transaction  history 
records  with  type  OCCR  Code  7,  8,  and  9  for  the  following: 

a.  Type  account  code  K  (ammunition) 

b.  Budget  Code  G  (fuels) 

c.  Budget  Code  I  (AEIC) 

d.  DIC/TRIC: 

(1)  A2 ( X)  Redistribution  Order 

(2)  A4 ( X )  Referral  Order 

(3)  FK( X)  Billings  Transactions 

(4)  FM( X )  File  Maintenance 

(5)  FT ( X)  Materiel  Return  Transactions 

(6)  SH ( X )  Non-Directed  Shipment 

(7)  SM( X )  IMR  Monetary  Adjustments 

(8)  REC  with  FIA  Code  Unequal  to  "440"  (Local  Manufacture 
Receipt ) 

(9)  TIN  if  Credit  Code  Equal  to  "F" 

2.  Step  2:  Further  select  the  transaction  history  records  from 
step  1  to  exclude  all  transaction  history  records  with  FIA  codes 
unequal  to  the  following: 

a.  Issue  FIA  Codes  -  310  317 

311  318 

312  319 

313  330 

314  331* 

315  334 

316  572 

*  NOTE:  Exclude  if  budget  code  unequal  "1". 

b.  Turn-in  FIA  Codes  -  420  427 

421  428 

422  429 

423  440 

424  441* 

425  664 

426  681 

*  NOTE:  Exclude  if  budget  code  Unequal  "1". 
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3.  Step  3:  Further  select  the  transaction  history  records  from 
step  2  for  "SRD"  equal  to:  SM(X)  1  (XX) 

SP(X)  2  (XX) 

A  (XX)  3  (XX) 

B  (XX)  4  (XX) 

C  (XX)  5  (XX) 

E  (XX)  6  (XX) 

F  (XX)  7  (XX) 

J  (XX)  8  (XX) 

K  (XX)  Q  (XX) 

X  (XX)  U  (XX) 


Al-3 


V  ■ 


I 


Attachment  2:  Proposed  DAR  Entries  Supporting  Correction  of 
Programming  Error  in  D002A 

Requirement: 

Supply  Consumable  Material  data  is  provided  to  the  Component 
Support  Cost  System  (DSD  D160B)  by  D002A  in  accordance  with  pro¬ 
cedures  described  in  AFM  177-206,  Chapter  60.  Personnel  of  the 
Air  Force  Data  Systems  Design  Center  have  identified  (D  a 
programming  error  which  they  describe  as  follows: 

"Local  manufacture  transactions  (TRIC  "REC"  with  FIA  440)  are 
assumed  to  be  issues  by  the  program  instead  of  a  turn-in.  The 
end  result  is  a  local  manufacturer  receipt  and  an  issue  can 
cause  identical  contributions  to  the  Daily  CSCS  Record  (1CM)." 

Impact  Statement 

Failure  to  implement  means  that  CSCS  base  material  costs  will 
be  in  error  by  twice  the  value  of  local  manufacture  receipts. 

Justification  Benefits/Cost  Savings 

Required  to  correct  an  acknowledged  programming  error. 

Although  the  impact  of  the  error  on  the  CSCS  as  a  whole  is  a 
small,  material  cost  outputs  are  now  wrong  for  any  Work  Unit  Code 
with  any  significant  amount  of  local  manufacture  receipts. 


Letter  from  Chief,  Material  Systems  Division,  Directorate  of 
Comptroller  Systems,  Air  Force  Data  Systems  Design  Center,  to 
HQ  AFLC/MM  (VAMOSC),  dated  15  September  1983.  Subject:  D002A, 
Daily  Consumable  Material  Cost  Data  Interface  with  D160B, 
Component  Support  Cost  System  (CSCS)  (Your  ltr,  15  Aug  83)." 
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